Demographic Bias Analysis

Demographic Bias Analysis Key Points

Why is bias important to consider in your models?

It's important to note that bias within models can restrict or limit patient access to key medical benefits from government aid programs.

Programs using AI algorithms to help automate approvals of key government benefits are becoming more commonplace. However, it is just as important to consider how bias can unintentionally occur.

Another reason that you want to consider bias in models is that in order to create better treatments for patients we need to find better ways to select and recruit patients that represent the wider population that a drug/treatment would be targeted for.

In many cases, there may be systemic biases for key groups and while this cannot always be prevented, bringing awareness of limitations and biases can give a more accurate picture of a treatment's effectiveness across different demographics.

In the example above, we can see that the purple parallelogram is representative of the population, while the blue circles and triangles are not and both are over-represented in the trial.

Unintended Bias

Unintended biases: a bias that is not intentional and often is not even apparent to the creator of a model

Unintended biases represent the unconscious or unintentional biases that come with the AI models that we are building. Becoming more aware of these biases and how they impact different groups is key.

Note: We usually associate bias with a negative connotation, but biases can be a source of valuable prior information. The problem can be when we are not aware of our biases and do not account for those that have a significant impact on the populations these models serve.

Aequitas

• Developed by University of Chicago Data Science for Social Good
• Addresses concerns about unintended bias unfairly affecting certain groups
• Definitions and metrics for unintended bias in predictive models